Reliability Estimates for Regression Predictions: Performance Analysis
نویسندگان
چکیده
In machine learning, the reliability estimates for individual predictions provide more information about individual prediction error than the average accuracy of predictive model (e.g. relative mean squared error). Such reliability estimates may represent decisive information in the risk-sensitive applications of machine learning (e.g. medicine, engineering, and business), where they enable the users to distinguish between more and less reliable predictions. In the atuhors’ previous work they proposed eight reliability estimates for individual examples in regression and evaluated their performance. The results showed that the performance of each estimate strongly varies depending on the domain and regression model properties. In this paper they empirically analyze the dependence of reliability estimates’ performance on the data set and model properties. They present the results which show that the reliability estimates perform better when used with more accurate regression models, in domains with greater number of examples and in domains with less noisy data. DOI: 10.4018/978-1-60960-537-7.ch014
منابع مشابه
Automatic selection of reliability estimates for individual regression predictions
In machine learning and its risk-sensitive applications (e.g. medicine, engineering, business), the reliability estimates for individual predictions provide more information about the individual prediction error (the difference between the true label and regression prediction) than the average accuracy of predictive model (e.g. relative mean squared error). Furthermore, they enable the users to...
متن کاملAutomatic Selection of Reliability Estimates for Individual Regression Predictions Using Meta-Learning and Internal Cross-Validation
In machine learning and its risk-sensitive applications (e.g. medicine, engineering, business), the reliability estimates for individual predictions provide more information about the individual prediction error than the average accuracy of predictive model (e.g. relative mean squared error). Furthermore, they enable the users to distinguish between more and less reliable predictions. The empir...
متن کاملComparison of approaches for estimating reliability of individual regression predictions
The paper compares different approaches to estimate the reliability of individual predictions in regression. We compare the sensitivity-based reliability estimates developed in our previous work with four approaches found in the literature: variance of bagged models, local cross-validation, density estimation, and local modeling. By combining pairs of individual estimates, we compose a combined...
متن کاملTowards Reliable Reliability Estimates for Individual Regression Predictions
In machine learning, the reliability estimates for individual predictions provide more information about individual prediction error than the average accuracy of predictive model (such as relative mean squared error). Individual reliability estimates may represent a decisive information in risk-sensitive applications of machine learning (e.g. medicine, engineering, business), where they enable ...
متن کاملEstimation of Individual Prediction Reliability Using Sensitivity Analysis of Regression Models
The dissertation [1–3] discusses the reliability estimation of individual regression predictions in the field of supervised learning. In contrast to the average measures for the evaluation of model accuracy (e.g. mean squared error), the reliability estimates for individual predictions can provide additional information which is beneficial for evaluating the usefulness of the predictions. This ...
متن کامل